Tamper detection for identification documents

ABSTRACT

Methods for detecting digital or physical tampering of an imaged physical credential include the actions of: receiving a digital image representing a physical credential having one or more high value regions, the digital image including an array of pixels; processing the digital image with a tamper detector to generate an output corresponding to an intrinsic characteristic of the digital image, the tamper detector configured to perform a pixel-level analysis of the high value regions of the digital image with respect to a predetermined tampering signature; and determining, based on the output from the tamper detector, whether the digital image has been digitally tampered with.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/651,011, filed Feb. 14, 2022, which is a continuation of U.S. patentapplication Ser. No. 16/741,465 (now U.S. Pat. No. 11,250,285), filedJan. 13, 2020, which is a continuation of U.S. patent application Ser.No. 15/783,311 (now U.S. Pat. No. 10,534,971), filed Oct. 13, 2017,which claims the benefit under 35 U.S.C. § 119(e) of U.S. PatentApplication No. 62/408,531, filed Oct. 14, 2016, each of which areincorporated herein by reference in their entireties.

TECHNICAL FIELD

This specification generally relates to systems and methods fordetecting tampered identification documents.

BACKGROUND

The use of physical identification documents has been pervasive invarious industries for decades. Moreover, in recent years, digitalimages of identification documents are increasingly being used forconducting secure, electronic transactions. Current techniques forauthenticating imaged identification documents involve systemsconfigured to scrutinize certain security features physically embeddedinto the underlying document. These security features are, by design,extremely difficult to replicate, and therefore effectively thwartattempts to produce counterfeit copies of the document. Many securityfeatures currently in use include intricate print patterns, digitalwatermarks, micro-printed text, unique emblems or logos, holograms, andthe like. Conventional authentication techniques for processing theseimaged identification documents is performed by systematically decodinginformation from a digital watermark and/or employing text or patternmatching techniques to verify the legitimacy of one or more othersecurity features.

SUMMARY

This specification describes technologies for detecting tamperedidentification documents based on digital images. More specifically,certain embodiments described herein involve techniques for detectingthe digital or physical tampering of identification documents based onone or more aspects that are intrinsic to a digital image, and not, forexample, associated with extracted text (e.g., text identified byoptical character recognition) or other encoded data (e.g., data encodedin security features or machine readable zones). Such aspects includepixel features that evidence physical and/or digital tampering, as wellas certain benign pixel features that include, but are not limited to:environmental, capture device, credential wear, lighting effects,hardware/software quantization, and/or digital compression effects. Insome examples, these tamper-detection techniques are applied to one ormore specific regions of interest—e.g., high value identificationregions of the identification documents.

Digital images of identification documents, as discussed in thisdisclosure, are digital images of physical credentials suitable for usein electronic transactions. The term “electronic transactions” refersbroadly to any computer-facilitated exchange between a possessor of aphysical or imaged identification document and one or more thirdparties. Electronic transactions can be conducted in-person or remotelyvia computer networks. Some electronic transactions may include theexchange of currency, but others may not. Suitable identificationdocuments for conducting secure, electronic transactions may include,but are not limited to, personal identity, employment, or professionalcredentials or certifications, or other high value identity-assertiondocumentation—e.g., a driver's license or passport. Further, in someimplementations, suitable identification documents may include so-called“breeder documents” (e.g., birth certificates, marriage certificates,social security documents, as well as utility bills, service bills, andother vital data correlation documents). The terms “identificationdocument” and “physical credential” may be used interchangeablythroughout this disclosure when referring to any document designed foridentity certification, assertion, or authorization that includesidentification data. The “identification data” may include one or moreof the following: an identification photograph, biographical information(e.g., a date of birth, an identification serial number, a socialsecurity number, a physical or electronic mailing address, a height, eyecolor, and gender), and/or one or more machine readable zones (MRZs)(e.g., a barcode or a Q-code). In some implementations, theidentification data may further include other biometric information inaddition to the ID photo, such as fingerprints, hand geometry, retinapatterns, iris patterns, handwriting patterns, and/or other physicalmorphological identity characteristics. Regions of the imagedidentification document that contain this identification data arereferred to generally as “high value regions” throughout the presentdisclosure because of their importance in identifying the document'spossessor in an electronic transaction.

One or more embodiments of the present disclosure are resultant of arealization that conventional techniques for authenticating imagedidentification documents are difficult to implement, prone to failure,and/or suffer from severe security vulnerabilities. As one example,authentication techniques reliant upon security features can bedifficult to implement on a large scale because they requiremodifications to the physical identification documents. This amounts toa reissuance of the credential to each possessor. These modificationscan take a long time to propagate through a large universe of physicalcredentials, such as passports and driver's licenses, because users tendto replace them infrequently. Thus, for instance, it could take years tofully implement a digital watermarking system that requires coded datato be embedded in each document. These conventional authenticationtechniques can also be prone to failure because the decoding and/ortext/pattern recognition routines require the identification document tobe imaged in very particular lighting conditions and/or alignmentorientations. It often takes the user several attempts to achieve asuitable image capture. More importantly, while conventional securityfeatures can be effective at inhibiting or preventing successfulcounterfeiting, they are not helpful in detecting whether anauthentically issued physical identification document has been digitallyor manually tampered with. For example, the possessor of an authenticidentification document may tamper with that document by replacing oraltering certain high value regions (e.g., photos, biometrics,biographics, and MRZs) that are critical for identifying the possessorin electronic transactions.

This type of tampering can often be achieved without affecting theembedded security features (e.g., where the security features do notoverlay the high value regions of the identification document), and thuswill not be detected by conventional authentication techniques, whichallows the document possessor to hide or outright replace criticalinformation in order to conceal his/her identity. Moreover, it isrelatively simple to manipulate non-security feature aspects of theidentification document, including the high value regions, usingcommercially available image editing software. Of course, attempts attampering with identification documents tend to vary in type and levelof sophistication. At the lower sophistication levels, entire regions ofthe identification document may be altered or replaced (digitally orphysical) without making any attempts to match texture or font. Otherattempts may be more refined. For example, the forger may utilizespecial software in an attempt to meticulously recreate backgrounds,security features, and the like. As yet another example, the forger mayattempt to homogenize the modified portions of the image by taking a newlive photo of a printout or screenshot of the splice or tamper. Theseand a myriad of other tamper techniques can be used to effectivelyundermine conventional authentication methods.

Accordingly, embodiments of the present disclosure aim to resolve theseand other problems with conventional authentication techniques byproviding a fundamental paradigm shift in the field that does not relysolely on security features to verify the legitimacy of imagedidentification documents. In particular, the present disclosure relatesto techniques for validating the intrinsic image characteristics withinand around certain high value regions of the imaged identificationdocument. As will be evident in view of the following discussion, theunconventional techniques of this disclosure are somewhat agnostic tosecurity features insofar as these features are not scrutinized based onpredetermined patterns or encoded data. Further, it is understood thatin certain types of identification documents, the embedded securityfeatures may partially or entirely overlay the high value regions ofinterest. In cases where no such overlay exists, the techniquesdescribed herein may serve as the only assurance of document integrity.In overlay cases, the presently described techniques provide additionalvalue to those security features by ensuring that they are native to thedocument and not forged over the modified high value region(s) or liftedfrom another document of the exact same type.

In general, one innovative aspect of the subject matter described inthis specification can be embodied in methods that include the actionsof: receiving a digital image representing a physical credential havingone or more high value regions, the digital image including an array ofpixels; processing the digital image with a tamper detector to generatean output corresponding to an intrinsic characteristic of the digitalimage, the tamper detector configured to perform a pixel-level analysisof the high value regions of the digital image with respect to apredetermined tampering signature; and determining, based on the outputfrom the tamper detector, whether the digital image has been digitallytampered with.

Other embodiments of this aspect include corresponding computer systems,apparatus, and computer programs recorded on one or more computerstorage devices, each configured to perform the actions of the methods.For a system of one or more computers to be configured to performparticular operations or actions means that the system has installed onit software, firmware, hardware, or a combination of them that inoperation cause the system to perform the operations or actions. For oneor more computer programs to be configured to perform particularoperations or actions means that the one or more programs includeinstructions that, when executed by data processing apparatus, cause theapparatus to perform the operations or actions.

The foregoing and other embodiments can each optionally include one ormore of the following features, alone or in combination. In particular,one embodiment includes all the following features in combination.Further, in some examples, the pixel-level analysis includes examinationof a combination of multiple intrinsic pixel features. In some examples,the combination of intrinsic pixel features includes: one or more pixelfeatures that evidence physical and/or digital tampering; and one ormore benign pixel features including: environmental, capture device,credential wear, lighting effects, hardware/software quantization,and/or digital compression effects.

In some examples, a high value region of the received digital imageincludes biographical text, and the predetermined tampering signature ofthe tamper detector includes biographics splicing or other modification.In some examples, a high value region of the received digital imageincludes biometric data, and the predetermined tampering signature ofthe tamper detector includes photo splicing or other modification. Insome examples, the biometric data includes an identification photograph.In some examples, a high value region of the received digital imageincludes a machine readable zone (MRZ), and the predetermined tamperingsignature of the tamper detector includes MRZ splicing or othermodification. In some examples, the predetermined tampering signature ofthe tamper detector includes image blurring or occlusion of one or moreof the high value regions.

In some examples, the tamper detector includes a predictive modeltrained by a machine learning algorithm applying a training dataset. Insome examples, the training dataset includes: a first set of untampereddigital images; and a second set of digital images including digitallytampered derivations of one or more of the untampered digital images.Each of the untampered digital images of the first set are assigned afirst common label in the training dataset, and each of the tampereddigital images of the second set are assigned a second common label inthe training dataset, the second label differing from the first.

In some examples, the first set of untampered digital images is selectedto induce examination of a combination of multiple intrinsic pixelfeatures. In some examples, the combination of intrinsic pixel featuresincludes: one or more pixel features that evidence physical and/ordigital tampering; and one or more benign pixel features including:environmental, capture device, credential wear, lighting effects,hardware/software quantization, and/or digital compression effects. Insome examples, the first set of untampered digital images includes aplurality of images representing different forms of personal identitycredentials. In some examples, the different forms of personal identitycredentials include different densities of security features. In someexamples, the different forms of personal identity credentials includedifferent layout designs. In some examples, the first set of untampereddigital images includes a plurality of images captured under differentlighting conditions. In some examples, the first set of untampereddigital images includes a plurality of images captured with differentimaging devices. In some examples, the first set of untampered digitalimages includes a series of digital images representing the samephysical credential, each digital image of the series oriented at adifferent degree of rotation. In some examples, the first set ofuntampered digital images includes a plurality of randomly rescaledimages. In some examples, the first set of untampered digital imagesincludes a plurality of images including randomly introduced digitalnoise. In some examples, the second set of digitally tampered images arederived by automatically modifying one or more portions of the pixelarray of each untampered image of the first set, the modificationdetermined based on the tampering signature of the tamper detector. Insome examples, the second set of tampered images are derived frommanually modifying physical credentials with physical tampering methodsfollowed by digitally imaging a physically modified credential. In someexamples, the second set of physically modified tampered images arefurther modified at the pixel level with digital editing tools to maskthe physical tamper. In some examples, the second set of tampered imagesare derived from manually or automatically produced physical or tampersupon physical credentials with physical use characteristics beingvisually evident. In some examples, the visually evidence usecharacteristics include wear, damage, design flaws, or deliberatephysical modification. In some examples, the second set of tamperedimages are derived from original uncorrected images with lighting, angleto the focal plane, skew, rotation, blur inherent from the originaldigital imaging event. In some examples, the second set of tamperedimages are derived from enhanced corrected images which are angled tomatch the document print orientation, either vertical or horizontal andoriented to 0 or 90 degrees on an edge, skew corrected to proper aspectratio, blur and lighting and other effects corrected and enhanced beforetamper.

In some examples, a plurality of digital images of the training datasetinclude a pixel array having a modified principal component. In someexamples, the predictive model includes a probabilistic classifier, andthe output includes a classification of the digital image and a degreeof certainty. In some examples, determining whether the digital imagehas been tampered with includes comparing the degree of certainty to apredetermined threshold value.

In some examples, the predictive model includes a convolutional neuralnetwork classifier having a multi-layer node-based architecture, theoutput layer of the classifier including a multi-class data layer.

In some examples, analyzing the pixel array includes the actions of:identifying a region of interest as a subset of the pixel array; andimplementing a tamper detection algorithm with respect to only theidentified region of interest tuned and trained for that region ofinterest. In some examples, analyzing the pixel array includes theactions of: identifying a region of interest as a subset of the pixelarray; and implementing multiple tamper detectors based on differenttampering signatures with respect to the identified region of interest.In some examples, determining whether a digital image has been tamperedwith includes applying multiple independent thresholds to respectiveoutputs from the multiple tamper detectors. In some examples, analyzingthe pixel array includes the actions of: identifying multiple regions ofinterest as subsets of the pixel array; and implementing a uniquecombination of one or more different tamper detection algorithms withrespect to each of the regions of interest. In some examples,determining whether a digital image has been tampered with includes theactions of: for each region of interest, applying multiple independentthresholds to respective tamper outputs from the one or more tamperdetectors to determine multiple tamper probabilities; for each region ofinterest, aggregating the multiple tamper probabilities to determine aregion-level tamper probability, and aggregating the region-level tamperprobabilities of the multiple regions to determine a document-levelprobability of tamper.

In some examples, the physical credential represented by the receiveddigital image includes one or more physically embedded securityfeatures, and the tamper detector is further configured to perform thepixel-level analysis agnostic of the security features embedded in therepresented physical credential.

The details of one or more embodiments of the subject matter of thisspecification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example system for generating an imageclassifier.

FIG. 2 is a flowchart of an example process for training an imageclassifier.

FIG. 3 is a flowchart of an example process for generating trainingimages.

FIG. 4 is a process diagram illustrating a training phase for generatingan image classifier.

FIG. 5 is a process diagram illustrating a test phase for evaluating animage classifier.

FIGS. 6A-6F are illustrative diagrams of natural and tampered digitalimages of a state driver's license.

FIGS. 7A-7F are illustrative diagrams of natural and tampered digitalimages of a national passport.

FIG. 8 is a diagram of an example system for providing output indicatingwhether an imaged identification document has been tampered with.

FIG. 9 is a flowchart of an example process for providing outputindicating whether an imaged identification document has been tamperedwith.

FIG. 10A is a diagram comparing tampered and untampered identificationdocuments represented by digital images.

FIG. 10B is a diagram illustrating various intrinsic layers of adigitally imaged identification document.

Like reference numbers and designations in the various drawings mayindicate like elements.

DETAILED DESCRIPTION

One or more embodiments of the present disclosure involve systems andmethods for detecting tampered identification documents that have beenimaged for conducting electronic transactions. In particular, certainembodiments may include processing a digital image of a physicalidentification document with a tamper detector to generate an outputcorresponding to an intrinsic characteristic of the digital image. Thetamper detector may perform a pixel-level analysis of the digital imagewith respect to a predetermined digital and/or physical tamperingsignature. The term “pixel-level analysis” refers to thecomputer-implemented examination of individual pixels, or small groupsof pixels, in a digital image. For instance, the pixel-level analysismay examine certain pixel features that evidence physical and/or digitaltampering, as well as certain benign pixel features that include, butare not limited to: environmental, capture device, credential wear,lighting effects, hardware/software quantization, and/or digitalcompression effects. These aspects are discussed in detail below. Theterm “tampering signature” refers to the specific type(s) of digital orphysical tampering that the tamper detector is configured todetect—e.g., image splicing, MRZ splicing, biographics splicing, and/orblurring/occlusion. In some examples, the tamper detector is configuredto perform the pixel-level analysis substantially agnostic of thespecific security features embedded in the represented physicalcredential. That is, while the tamper detector may examine pixels of theimage that represent the security features (e.g., when the securityfeatures overlay certain high value regions of the identificationdocument), it may not implement any specific decoding or patternmatching routines to verify them.

In some embodiments, the tamper detector includes a trained imageclassifier. Image classifiers are generally used to classify imagesaccording to predefined labels. An image classifier may identify apredefined label that matches or partially matches an image, and thenassociate the identified label with the image. For example, an image ofa cat sitting on a chair may be labeled “cat,” “chair,” or both. In someimage classifiers, an image may be labeled according to broad categoriesof image content, e.g., animal or human. In other image classifiers, animage may be labeled according to narrow categories, e.g., Persian catsor Bengal cats. Certain embodiments of the present disclosure involvethe construction of an image classifier specifically configured todetermine whether the high value regions of an identification documenthave been tampered with. For example, the image classifier may label theimage of the identification documented as “tampered” or “untampered”based on a number of intrinsic features obtained from a pixel-levelanalysis of the digital imaged identification document. The imageclassifier can be trained based on one or more features extracted from acollection of training images. The training images are representativeexamples of various imaged identification documents that are eithertampered or untampered, and labeled as such. As discussed throughoutthis disclosure, the selection/creation and labeling of training imagesis specifically designed to optimize and/or reinforce certain weights inthe classifier architecture based on one or more aspects of theintrinsic features in these images with respect to the high valueregions data included in the underlying identification documents.

In some examples, a machine learning system may be used to processtraining data and construct suitable image classifiers for enabling thetechniques described herein. These machine learning systems may receiveand process a myriad of unique training examples, each of which embodiesa distinct combination of intrinsic features that are separate and apartfrom the security features embedded within the physical credential. Whendeveloping an image classifier, various architectures may be used,including, for example, a neural network. In certain implementationsdescribed herein, a Convolutional Neural Network (“CNN”) or DeepConvolutional Neural Network (“DCNN”) may be used.

FIG. 1 is a diagram of an example system 100 for generating an imageclassifier 106. The image classifier 106 may be appropriately configuredto distinguish between tampered and untampered digital images ofphysical credentials based on intrinsic features associated with certainhigh value regions of identification data. As shown, the system 100includes a classifier generator 104 that receives a collection oftraining images 102. The classifier generator 104 utilizes the trainingimages 102 to generate an image classifier 106. The collection oftraining images 102 includes a group of positive digital images labeledas “tampered” and a group of negative digital images labeled as“untampered”. The positive and negative (tampered and untampered)digital images can be labeled manually by human users, or automaticallyby a training data generation system. Specific techniques for developingthe collection of training images 102 are described in greater detailbelow with respect to FIGS. 6A-7F.

In this example, the classifier generator 104 includes a featureextraction engine 108, feature instances 110, and a classifier trainingengine 112. The feature extraction engine 108 extracts intrinsicfeatures (e.g., intrinsic pixel features) from the received trainingimages 102. In some examples, the feature extraction engine 108 mayinclude a CNN (e.g., a pre-trained CNN generated from a large collectionof diverse images). A number of different types of features areextracted as indicative of a digital image of physical credential. Forexample, the extracted features can include textual information, edgesof photos, patterns of photos, etc. The extracted features from eachdigital image in the received training images 102 form a collection offeature instances 110. These extracted features are used by theclassifier training engine 112 to train an image classifier. Not allfeatures need be provided to the classifier training engine 112.Instead, different combinations of features can be used depending on theparticular classifier application—e.g., the intended tampering signatureof the trained image classifier 106.

In some examples, extracted intrinsic pixel features may include one ormore aspects relating to image capture lighting. Lighting aspects mayinclude imaging effects from different configurations of light sources(multiple or single), light color/wavelength responses, angles ofincidence, etc. In some examples, extracted pixel features may includeone or more aspects relating to selected image capture devices. Aspectsrelating to the image capture devices may include effects (e.g.,pixilation/resolution, noise, manufacturing defects) from usingdifferent types of image sensors (e.g., charge-coupled devices (CCD),complementary metal-oxide-semiconductor (CMOS), or N-typemetal-oxide-semiconductor (NMOS)). In some examples, extracted pixelfeatures may include one or more aspects relating to optical lensing.Optical lensing aspects may include effects such as fixed/variable focallength, fisheye, as well as other optical lensing distortions (e.g.,lens damage/dirt). In some examples, extracted pixel features mayinclude one or more aspects relating to certain imaging environments.Imaging environment aspects may include color/intensity/hue response bydifferent image sensors across varying backgrounds and/or foregrounds.Imaging environment aspects may also include multiple/single focalpoints/planes with physical/digital estimation, bisection of the targetfocal plane, and effects of foreground/background in the focal plane. Insome examples, extracted pixel features may include one or more aspectsrelating to hardware and/or software quantization. Quantization aspectsmay include imaging effects produced by colorspace rendering inquantized space from continuous physical space, quantization andestimation by various imaging libraries, as well as furtherquantization/compression by compressed image formats. In some examples,extracted pixel features may include one or more aspects relating toautomatic software-implemented post-imaging enhancements, such as: colorbalancing, image enhancement, histogram equalization, multiplecolorspace/response, etc. In some examples, extracted pixel features mayinclude one or more aspects relating to manual post-imagingmodifications, such as: filters, transforms, and the like.

Various aspects relating to extracted intrinsic pixel features arefurther exemplified by a discussion of FIGS. 10A and 10B. FIG. 10Aprovides a comparison of three images—a first untampered image 1000 a ofa first driver's license, a second untampered image 1000 b of a seconddriver's license, and a third image 1000 c that has been tampered byreplacing or “splicing” the photo and biographic regions of the firstimage 1000 a with the corresponding regions of the second image 1000 b.FIG. 10B provides a layered illustration of the third image 1000 c.

In the example of FIG. 10B, “Layer Zero” (1002) represents the originalperfect document as produced by the original manufacturer. This layerincludes the underlying identification document, with all the visiblelight security features, high value regions (biometrics, biographics,MRZs, etc.) as if it was perfectly imaged. “Layer One” (1004) representsthe dirt and damage that can accumulate over time. This may include, forexample, laminate quality, use, wear and tear, etc. This layer may alsorepresent the laminate or outer surface of the credential itself,including reflectivity (e.g., degree of gloss or perforation) and damage(e.g., wear and tear, rubbing, cracking, color fade from sun exposure ordirt). This layer may also represent deliberate physical modification ofthe credential including cutting, perforation, or the affixing ofstickers or other post manufacturing materials to the surface (e.g.,change of address stickers, perforation by an agency due to expirationand replacement, and deliberate physical tampering to misrepresentdata). In general, this layer corresponds to one or more physicalaspects of the underlying document.

“Layer Two” (1006) represents the effects of lighting and environmentduring the image capture event. Lighting effects may relate to thenumber of light sources, the degree of light intensity and angle (e.g.,lighter towards the light source, darker further away), the presence ofan optically variable device (“OVD”), such as a color changing securityfeature/hologram, the degree of light color/hue/intensity, etc.Environment effects may relate to contrast between the imaged documentand the background, as well as other similar aspects. “Layer Three”(1008) represents lensing effects and focal length. For example, fisheyemay occur when the document is too close to the lens or positioned atangle relative to the lens. These imperfections in orientation at thetime of image capture may also cause problems with respect to the focallength, resulting in portions of the document that are further or closerto the lens to become partially blurry. In general, this layercorresponds to areas that may be in or out of focus based on theconditions of image capture. “Layer Four” (1010) represents effectsrelating to exposure, such as color/lighting response of the CCD,shutter speed (e.g., exposure duration), and the like. In general, thislayer corresponds to areas that may appear “washed out” do to exposureimperfections during image capture. “Layer Five” (1012) representseffects relating to the CCD (or other imaging sensors)—the manufacturingquality, for example. Low quality imaging sensors tend to cause portionsof the image to appear cloudy, noisy, washed out, or otherwisedefective. This layer may also capture differences in scan and refreshrates between computer screens and imaging sensors resulting inpixelization and moire colorspace effects from wavelength harmonics. Ingeneral, this layer corresponds to overall uniformity in defectstypically caused by imperfect imaging sensors and the physicalwavelength harmonics between sensors and screens.

“Layer Six” (1014) represents effects relating to hardware and/orsoftware quantization. As will be appreciated by those of skill in theart, the purpose of quantization is to effectively transform aspects ofthe real world (e.g., colors, shadows, lights, etc.) into pixels. FIG.10B provides an example of the quantization effect in the image of“Layer Six” (1014) where there is a significant difference pixilationbetween the left and right side of the image. In general, this layercorresponds to the wide range of pixilation that may occur at the timeof image capture. “Layer Seven” (1016) represents effects from softwareimage enhancement and compression. These aspects typically relate tore-quantizing to the available color palette and depth (e.g., 8 bit or24 bit color). Enhancements may also include histogram equalization, andother filters and effects to boost dark areas and soften lighter ones.Enhancement may be followed by compression, which can again re-quantizeto save space, reducing pixel count, reducing depth, etc. In general,this layer accounts for groups of pixels that are indicative of higherpixilation levels from post-capture processing.

The seven layers described above represent exemplary intrinsic pixelfeatures that may be perceived by image classifiers (as well as othertypes of tamper detectors) and identified as associated with natural oruntampered imaged identification documents. That is, the intrinsicfeatures described above tend to be induced by routine and conventionalimage capture and processing operations, and are therefore benign. Onthe other hand, “Layer Eight” (1018) represents aspects of a tamperedimage (i.e., image 1000 c of FIG. 10A) that can be distinguished fromother the benign intrinsic features of the prior seven layers. Intrinsicfeatures specific to this eighth layer are associated withidentification documents that have been physically or digitally tamperedwith. Various embodiments of the present disclosure are directed totechniques for training and applying image classifiers configured toidentify and distinguish between intrinsic pixel features that arebenign and those that evidence a tampering attack.

Referring back to FIG. 1 , any suitable type of image classifier can betrained and used to classify digital images of physical credentials. Forexample, the image classifier 106 may be a CNN, a support vectormachine, or a Bayesian classifier. In any event, training the imageclassifier 106 by the classifier training engine 112 involves a machinelearning process that learns respective weights to apply to each inputfeature. Typically, an image classifier is trained using an iterativeprocess that progressively develops weights for each of the inputfeatures. That is, the image classifier may include default or initialweights assigned to each feature that are iteratively amended as theprocess attempts to find optimal weights. Based on the learned featureweights, an input digital image of a physical credential can be scoredby the generated image classifier 106. In some implementations, eachinput digital image of a physical credential is scored by the imageclassifier 106 as a probability of being tampered on a scale from0.0-1.0. In some implementations, the output classifier score for eachdigital image of a physical credential is compared to a thresholdprobability value to determine whether an imaged physical credential hasbeen tampered with (digitally or physically) or is untampered. In someexamples, scores from multiple image classifiers can be combined toproduce an aggregate score, or the respective scores of each classifiermay be evaluated individually in series.

In some implementations, after training, the image classifier 106 can beevaluated using an obtained group of sample images. The sample imagesare distinct from the training images and have known tampered/untamperedlabels. Similar to the training images 102, the group of sample imagesmay include both positive/tampered digital images andnegative/untampered digital images. Each digital image of the group ofsample images are scored by the image classifier 106. As noted above,the score indicates a likelihood that the digital image of the physicalcredential is tampered or untampered. The scores are compared to theknown labels to evaluate the accuracy of the image classifier 106. Ifthe output of the image classifier 106 does not substantially match theknown labels, the classifier 106 can be adjusted or trained on a new setof training images. Further, in some examples, differences between theoutput of the image classifier 106 and the known labels during the testphase may affect the scoring thresholds used to label classificationresults (e.g., specific operational thresholds can be chosencorresponding to the score response of a given set of data).

FIG. 2 is a flowchart of an example process 200 for training an imageclassifier. The process 200, and all other processes described herein,can be performed by a system of one or more computers, located in one ormore locations, programmed in accordance with one or more embodiments ofthis disclosure. For example, the classifier generator 104 of FIG. 1 canperform the process 200.

According to process 200, the system obtains a collection of trainingimages (202). The collection of training images can be generated by thesystem or obtained from another source. The training images (e.g.,training images 102) include positive examples of digital images thatare labeled as tampered and negative examples of digital images that arelabeled as untampered or natural. The system extracts features from thetraining images (204). As described above, a feature extractor (e.g.,the feature extractor engine 108) can extract one or more intrinsicpixel features from the received training images. The system uses theextracted feature instances to train an image classifier having theextracted features as variables (206). The training of the imageclassifier assigns optimum weights to the respective features thatmaximize the likelihood of correctly labeling an input digital image astampered or untampered. The system evaluates the image classifier (208).In particular, the image classifier is applied to a group of knownsample images. The output of the image classifier is compared to theknown labels of the digital images. As a result, the image classifiercan be validated. The image classifier can then be applied to a largercollection of digital images and can be used for detecting an imagedphysical credential that has been tampered with.

FIG. 3 is a flowchart of an example process 300 for generating acollection of training images that can be used to train an imageclassifier (e.g., step 202 of process 200). According to the process300, the system obtains untampered digital images of physicalcredentials (302). The untampered digital images of physical credentialsare digital images that are not edited, spliced, or otherwiseintentionally tampered with (note, however, that untampered images canbe modified to mimic different lighting conditions, environmentaleffects, etc.). More specifically, the collection of untampered imagesis specifically selected to induce the identification, extraction, andweight optimization of certain intrinsic pixel features. For example,the selected intrinsic pixel features may relate to various aspects ofdigital images produced by commercial off the shelf (COT) imagingdevices. Thus, as discussed above, the untampered images may be selectedto achieve inspection of a combination of intrinsic features includingone or more of: image capture lighting, image capture devices, opticallensing, imaging environments, hardware and/or software quantization, orautomatic or manual post imaging enhancements. Various embodiments ofthe present disclosure are derivative of a realization that selectedcombinations of such intrinsic features comprise a unique “fingerprint”of a digital image. Indeed, it has been discovered that training aclassifier to identify and scrutinize the fingerprint of each imagedidentification document with respect to certain high value regionsproduces surprisingly accurate results in tamper detection.

In some examples, suitable untampered images may include digital imagesof driver's licenses for various states in the United States and foreigncountries, digital images of passports for various countries, or digitalimages of government identifications for various government departmentsand agencies can be untampered digital images of physical credentials.In addition, the untampered digital images include digital images ofvarious versions of a particular physical credential. For instance,digital images of California driver's licenses issued in 1990s, 2000s,and 2010s can be untampered digital images. Moreover, the untampereddigital images of physical credentials may include digital images of aparticular physical credential that are obtained under variouscircumstances. For instance, digital images of a particular driver'slicense that are taken under various lighting conditions or usingvarious image capture devices (e.g., point-and-shoot cameras, cell phonecameras, etc.). Example factors that affect different lightingconditions include color, temperature, light intensity, and a directionof incoming light. In some examples, the untampered digital images maypresent visually evidence physical use characteristics, such as wear,damage, design flaws, or deliberate physical modification. Further, insome examples, the untampered digital images may be “uncorrected” withrelative to the original imaging event. Thus, the untampered images mayinclude flaws with respect to lighting, angle to the focal plane, skew,rotation, and blur. In some other examples, the untampered images may beautomatically processed by the system to correct such flaws. Of course,numerous other operations for providing a suitable collection ofuntampered images to induce examination of certain intrinsic pixelfeatures are also contemplated within the scope of this disclosure.

Still referring to FIG. 3 , the system generates a first set of tamperedimages based on the untampered digital images (304). In someimplementations, the first set of tampered digital images can begenerated manually by a human user or automatically by the system. Forexample, the first set of tampered digital images may be generated by:(i) replacing an original photo in a digital image of a physicalcredential with a different photo (“photo splicing”), (ii) replacingoriginal biographic information in a digital image of a physicalcredential with different biographic information (“biographicssplicing”), (iii) replacing an original MRZ in a digital image of aphysical credential with a different MRZ (“MRZ splicing”), and/or (iv)occluding or blurring a photo, biographics, or the MRZ in a digitalimage of a physical credential. Examples of generating the first set oftampered digital images of physical credentials based on the untampereddigital images of physical credentials are described below with respectto FIGS. 6A-7F.

The system may optionally obtain a second set of tampered images ofphysical credentials (306). The system can obtain the second set oftampered images from sources other than the original untampered images(e.g., a repository of tampered images). For example, the second set oftampered images may include images (or variations thereof) that weremislabeled during testing or live use of the system. In any event, thesystem labels the first and (optionally) second sets of tampered imagesas “tampered” (308). In addition, the system labels the untampereddigital images of physical credentials as “untampered” (308). The systemmay also augment the first and second sets of tampered images, as wellas the untampered images (310). The augmentation of the digital imagesadvantageously increases and further diversifies the collection oftraining images, which improves the performance level of the imageclassifier. In some examples, the system can augment the entire digitalimage, or just a particular portion of a digital image. In addition, thesystem can apply different augmentation techniques on different portionsof a digital image.

The system can use various techniques for augmentation of theuntampered/tampered digital images. In some implementations, the systemcan augment digital images by resizing the digital images. For example,the system can enlarge or shrink a size of a digital image. In someimplementations, the system can augment digital images by rotating thedigital images. For example, the system can rotate digital images by 90,180, or 270 degrees (of course, other degrees of rotation may also beused—e.g., 5, 120, and 260 degrees). In some implementations, the systemcan augment digital images by adding noise, e.g., random Gaussian Noise,to the digital images. In some implementations, the system can augmentdigital images by changing photometric features. For example, the systemcan change colors in a digital image. Further still, in someimplementations, the system can augment digital images viare-quantization into different color depths and image channelarchitectures (1-channel, 3-channel, 4-channel etc.), compression usingvaried image compression formats, and/or a variety of other filteringeffects.

The system then generates a collection of training images based on theaugmented digital images (312). The training images generated by thesystem include two groups of digital images; one group of digital imagesis labeled as tampered and another group of digital images is labeled asuntampered. The training images can be provided to a classifiergenerator, e.g., the classifier generator 104 of FIG. 1 , for trainingan image classifier.

FIG. 4 provides an exemplary process diagram illustrating a trainingphase for generating an image classifier in accordance with processes200 and 300 described above. As shown in FIG. 4 , the training phasebegins by receiving natural or “untampered images.” The untamperedimages are then manually or automatically tampered as discussed belowwith reference to FIGS. 6A-7F. The tampered and untampered images arelabeled as such, and then processed by resizing (if necessary) andaugmentation. The augmented training images are then provided to the CNNfor generating a trained image classifier. FIG. 5 provides a processdiagram illustrating a test phase for evaluating an image classifier(e.g., step 200 of process 200). As shown in FIG. 5 , the test phase isimplemented by submitting a test image to the system and processing thetest image with a trained CNN image classifier. The classifier outputs atamper probability, which is compared to a predetermined threshold todetermine whether a tamper alert is raised. The classification oftampered/untampered is verified, and any mislabeled images are added tothe training dataset.

FIGS. 6A-7F represent various examples untampered and tampered imagedidentification documents. In particular, digital images 600 a and 700 aare examples of untampered identification documents including thefollowing high value regions: photos 602,702, biographic data 604,704,and MRZs 606,706. Digital images 600 b-f and 700 b-f are examples ofidentification documents where certain high value regions have beentampered. Digital images 600 a-f correspond to a state driver's license,and digital images 700 a-f correspond to a passport.

As noted above, the tampered digital images 600 b-f and 700 b-f may bederived from the untampered digital images 600 a and 700 a. As a firstexample, tampered digital images 600 b and 700 b can be generated byreplacing the photo 602,702 with a new and different photo 602′,702′. Asa second example, tampered digital images 600 c and 700 c can begenerated by replacing the original biographic information 604,704 withnew and different biographic information 604′,704′. As a third example,tampered digital images 600 d and 700 d can be generated by replacingthe original MRZ 606,706 with a new and different MRZ 606′,706′. As afourth example, tampered digital images 600 e and 700 e can be generatedby occluding one or more high value regions of the imaged identificationdocument. In image 600 e, for instance, photo 602″ and biographicinformation 604″ have been occluded by blacking out portions of thoseregions. Similarly, in image 700 e, biographic information 704″ and MRZ706″ have been occluded by black out. Of course, various other types ofimage occlusion that obstruct visual inspection of a high value regionalso fall within the scope of the present disclosure. As a fifthexample, tampered digital images 600 f and 700 f can be generated byblurring one or more high value regions of the imaged identificationdocument. In image 600 f, biographic information 604″′ has been blurred.In image 700 f, all three high value regions—photo 702″′, biographicinformation 704″′, and MRZ 706″′ have been blurred.

In some examples splicing (e.g., replacing a portion of the digitalimage), occluding, and blurring of photos, biographics, and MRZs can beperformed automatically by the system to generate a portion of thetraining dataset. For instance, the system may utilize a face detector,text detector, or other suitable image-analyzing technologies toidentify the high value regions on a particular imaged identificationdocument. The identified region may then be selectively tampered asdiscussed above. In some examples, only certain portions of the highvalue regions may be tampered with. For instance, a face detector may beconfigured to identify certain selective portions of a photo—e.g., theeyes, hair, nose or mouth. As another example, a text detector may beconfigured to identify certain biographic information—e.g., a driver'slicense number, a date of birth, or a document discriminator. In someexamples, multiple tampered images can be derived from a commonuntampered image. For example, different techniques for splicing,occluding, and blurring can be used to create different types oftampered images. In some implementations, different splicing techniquescan result in softened, hardened, or jagged edges. Different splicingtechniques may also involve gradient domain stitching and/or balanceadjustments. Similarly, different degrees of blurring and/or occlusionsof different size, shape, and color may be used to derive multipletampered images. Automatically generating several different types oftampered images is advantageous because it increases the size of thetraining dataset. This advantage is amplified when CNNs are used tofacilitate the determination of whether an image corresponds to atampered identification document, because these types of classifierscontinue to improve with increasing training examples. Moreover, theautomatically generated tampered images further diversify the trainingdata by mimicking different levels of sophistication in tampering. Forexample, a rudimentary attempt at splicing may produce jagged edges,while a more sophisticated attempt may produce smooth edges.

FIG. 8 is a diagram of an example system 800 for providing outputindicating whether an imaged identification document has been tamperedwith. The system 800 can be implemented in an internet, an intranet, orany other suitable wired or wireless client-server environment. Thesystem 800 provides an example architecture in which the systems,components, and techniques described below can be implemented.

As shown, a user 802 can interact with a tampering detection system 814through a user device 804. For example, the user device 804 may includea computer, a camera, a scanner, a video recorder, or a mobile device,e.g., a mobile phone, or a tablet, coupled to the tampering detectionsystem 814 through a local area network (LAN) or wide area network(WAN), e.g., the Internet. In some implementations, the tamperingdetection system 814 and the user device 804 can be one machine. Forexample, the user 802 can install a computer program or an applicationon the user device 804. The user device 804 generally includes a randomaccess memory (RAM) 806 and a processor 808.

The user 802 can submit a digital image 810 of a physical credential toa tampering detection engine 830 within a tampering detection system814. In this example, the digital image 810 of the physical credentialis transmitted through a network to the tampering detection system 814.The tampering detection system 814 can be implemented as, for example,computer programs running on one or more computers in one or morelocations that are coupled to each other through a network.

The tampering detection system 814 includes the tampering detectionengine 830. The tampering detection engine 830 includes an imageclassifier trained using training images of physical credentials. Insome implementations, the tampering detection engine 830 can include theimage classifier 106 trained by the classifier generator 104 describedabove with respect to FIG. 1 . The tampering detection engine 830receives the digital image 810 of the physical credential and analyzesthe digital image 810 using pixel-level image analysis techniques. Thetampering detection engine 830 can perform a pixel-level analysis on thedigital image 810 that includes an array of pixels. The tamperingdetection engine 830 extracts intrinsic pixel features from the digitalimage 810 and determines whether the digital image 810 is tampered oruntampered based on the extracted features. The tampering detectionengine 830 generates the tamper indication output 828 based on thedetermination as to whether the digital image 810 is tampered oruntampered.

The tamper indication output 828 generated by the tampering detectionsystem 814 is provided to the user device 804 for presentation to theuser 802. In some implementations, the tamper indication output 828 isprovided to a server system 824 for use in authenticating the user 802.For example, the server system 824 can be a server system of a financialinstitution. The server system 824 can use the tamper indication output828 to authenticate that the user 802 is an owner of a particularaccount. In addition, the server system 824 can be a server system of anemployer. The server system 824 can use the tamper indication output 828to authenticate that the user 802 is an employee of the employer.Moreover, the server system 824 can be a server system of a hotel. Theserver system 824 can use the tamper indication output 828 toauthenticate that the user 802 is a guest staying at the hotel. Variousother types of electronic transactions are also envisioned within thescope of the present disclosure. Note that in many implementations, thetamper indication output may be combined with further processingtechniques to ultimately authenticate a user. That is, a tamperindication output of “natural” or “untampered” may inform the systemthat data included in one or more high value regions of the imagedcredential is genuine, and has not been tampered with. With thisguarantee, the system may go on to extract and process information fromthe high value regions to authenticate the user in accordance withconventional authentication techniques and practices.

With respect to FIG. 8 , in some implementations, the tamperingdetection engine 830 can include multiple image classifiers. Forexample, the tampering detection engine 830 can include a first imageclassifier that determines whether a digital image of a physicalcredential is tampered or untampered based on an identification photoarea of the digital image. The first image classifier can be trainedusing the training images described above with respect to FIGS. 6B and7B. The tampering detection engine 830 can further include a secondimage classifier that determines whether a digital image of a physicalcredential is tampered or untampered based on a biographic informationarea of the digital image. The second image classifier can be trainedusing the training images described above with respect to FIGS. 6C and7C. The tampering detection engine 830 can further include a third imageclassifier that determines whether a digital image of a physicalcredential is tampered or untampered based on an MRZ in the digitalimage. The third image classifier can be trained using the trainingimages described above with respect to FIGS. 6D and 7D. The tamperingdetection engine 830 can further include a fourth image classifier thatdetermines whether a digital image of a physical credential is tamperedor untampered based on occlusion or blurring of an identification photoarea, a biographic information area, an MRZ, or a biometric informationarea of the digital image. The fourth image classifier can be trainedusing the training images described above with respect to FIGS. 6E-F and7E-F.

In some implementations, the tampering detection engine 830 can applyone or more of the multiple image classifiers in series. The output frommultiple classifiers can be applied to one or more thresholdsindependently or aggregated to produce a combined output. For example,the tampering detection engine 830 can serially apply the first imageclassifier and the third image classifier to the digital image 810. Thetampering detection engine 830 can be configured to generate the tamperindication output 828 as tampered if either the first image classifieror the third image classifier determines that the digital image 810 istampered. In some implementations, the tampering detection engine 830can apply various combinations of image classifiers based on a type of aphysical credential. That is, different types ofcredentials/identification documents may have different high valueregions that require analysis by different image classifiers. Moreover,in some implementations, the tampering detection engine 830 can applyvarious combinations of image classifiers based on a determination as towhether a digital image of a physical credential represents a front sideor a back side of the physical credential. As a first example, thetampering detection engine 830 may be configured to apply the first,second, third, and fourth image classifiers for an image of the frontside of the passport shown in FIGS. 7A-7F because a comprehensive set ofhigh value regions are present. As a second example, the tamperingdetection engine 830 may be configured to use the first, second, andforth image classifiers for an image of the front side of the driver'slicense shown in FIGS. 6A-C and 6E-F, which includes an MRZ only on thebackside.

FIG. 9 is a flowchart of an example process 900 for providing outputindicating whether an imaged identification document has been tamperedwith. In some examples, the tampering detection system 830 describedwith respect to FIG. 8 can perform the process 900. According to theprocess 900, the system receives a digital image of a physicalcredential (902). The system receives the digital image of the physicalcredential from a user device. The system determines whether thereceived digital image of the physical credential is tampered oruntampered (904). The system receives the digital image of the physicalcredential and analyzes the digital image using image analysistechniques. The system can perform a pixel-level analysis on the digitalimage that includes an array of pixels. The system extracts intrinsicfeatures from the digital image and determines whether the digital imageis tampered or untampered based on the extracted features. The systemgenerates the tamper indication output based on the determination as towhether the digital image is tampered or untampered. The system providesthe tamper indication output (906). In some implementations, the tamperindication output generated by the system is provided to the user devicefor presentation to the user. In some implementations, the tamperindication output is provided to a server system of a third party forauthenticating the user.

In this specification the term “engine” will be used broadly to refer toa software based system or subsystem that can perform one or morespecific functions. Generally, an engine will be implemented as one ormore software modules or components, installed on one or more computersin one or more locations. In some cases, one or more computers will bededicated to a particular engine; in other cases, multiple engines canbe installed and running on the same computer or computers.

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Embodiments of the subject matter described in thisspecification can be implemented as one or more computer programs, i.e.,one or more modules of computer program instructions encoded on atangible non-transitory storage medium for execution by, or to controlthe operation of, data processing apparatus. The computer storage mediumcan be a machine-readable storage device, a machine-readable storagesubstrate, a random or serial access memory device, or a combination ofone or more of them. Alternatively, or in addition, the programinstructions can be encoded on an artificially-generated propagatedsignal (e.g., a machine-generated electrical, optical, orelectromagnetic signal) that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus.

The term “data processing apparatus” refers to data processing hardwareand encompasses all kinds of apparatus, devices, and machines forprocessing data, including by way of example a programmable processor, acomputer, or multiple processors or computers. The apparatus can alsobe, or further include, special purpose logic circuitry, e.g., an FPGA(field programmable gate array) or an ASIC (application-specificintegrated circuit). The apparatus can optionally include, in additionto hardware, code that creates an execution environment for computerprograms, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, or acombination of one or more of them.

A computer program, which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, a script, or code, can be written in any form of programminglanguage, including compiled or interpreted languages, or declarative orprocedural languages; and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A program may, but neednot, correspond to a file in a file system. A program can be stored in aportion of a file that holds other programs or data, e.g., one or morescripts stored in a markup language document, in a single file dedicatedto the program in question, or in multiple coordinated files, e.g.,files that store one or more modules, sub-programs, or portions of code.A computer program can be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a data communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby special purpose logic circuitry, e.g., an FPGA or an ASIC, or by acombination of special purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special purpose microprocessors or both, or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read-only memory or a random accessmemory or both. The essential elements of a computer are a centralprocessing unit for performing or executing instructions and one or morememory devices for storing instructions and data. The central processingunit and the memory can be supplemented by, or incorporated in, specialpurpose logic circuitry. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto-optical disks, or optical disks. However, a computer need nothave such devices. Moreover, a computer can be embedded in anotherdevice, e.g., a mobile telephone, a personal digital assistant (PDA), amobile audio or video player, a game console, a Global PositioningSystem (GPS) receiver, or a portable storage device, e.g., a universalserial bus (USB) flash drive, to name just a few.

Computer-readable media suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending digital images to and receiving digital images from a devicethat is used by the user; for example, by sending web pages to a webbrowser on a user's device in response to requests received from the webbrowser.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back-end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front-end component, e.g., aclient computer having a graphical user interface, a web browser, or anapp through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back-end, middleware, or front-end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (LAN) and a widearea network (WAN), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits data, e.g., an HTML page, to a userdevice, e.g., for purposes of displaying data to and receiving userinput from a user interacting with the device, which acts as a client.Data generated at the user device, e.g., a result of the userinteraction, can be received at the server from the device.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinvention or on the scope of what may be claimed, but rather asdescriptions of features that may be specific to particular embodimentsof particular inventions. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedabove as acting in certain combinations and even initially be claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various system modulesand components in the embodiments described above should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In some cases, multitasking and parallel processing may beadvantageous.

What is claimed is:
 1. A computer-implemented method for detectingdigital or physical tampering of an imaged physical credential, themethod being executed using one or more processors and comprising:receiving a digital image representing a physical credential having oneor more high value regions, the digital image comprising an array ofpixels; processing the digital image with a tamper detector to generatean output corresponding to an intrinsic characteristic of the digitalimage, the tamper detector configured to perform a pixel-level analysisof the high value regions of the digital image with respect to apredetermined tampering signature; and determining, based on the outputfrom the tamper detector, whether the digital image has been digitallytampered with.